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(57) ABSTRACT 

A method and apparatus for encoding digital video utilizing 
a single pass variable bit rate encoding procedure. An 
encoding quality is set and the bit rate for encoding blocks 
or frames in the sequence of moving pictures is determined 
to achieve the selected quality according to the complexity 
of the video sequence. The bit rate is constrained by prede- 
termined upper and lower bit rate limits. 
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ONE-PASS VARIABLE BIT RATE MOVING 
PICTURES ENCODING 

FIELD OF THE INVENTION 

The present invention relates to a method and apparatus 
for encoding moving pictures. In particular, the present 
invention relates to a method and apparatus for performing 
variable bit rate control for a digital video encoder. 

BACKGROUND OF THE INVENTION 

Digital compression has been applied to moving pictures 
for the purposes of transmissions bandwidth or storage size 
reduction. One current art of such compression techniques 
can be derived from the ISO/IEC MPEG Standards, the 
ISO/IEC 11172-3 (MPEG-1), the ISO/IEC 13818-2 (MPEG- 
2) and the MPEG-2 TM5 (test model 5), developed by the 
Moving Picture Experts Group of the International Organi- 
sation for Standardization. The disclosures of those stan- 
dards documents are hereby expressly incorporated into this 
specification by reference. 

In a standard MPEG compliant video encoder, a sequence 
of moving pictures (e.g. video) is input to the encoder where 
it is compressed with a user defined target bitrate. The target 
bitrate is set according to the communication channel band- 
width in which the compressed video is to be transmitted, or 
the storage media capacity in which the compressed video 
sequence is to be stored. 

Several different forms of coding can be employed 
depending upon the character of the input pictures, referred 
to as I-pictures, P-pictures, or B-pictures. The 1-pictures are 
intra-coded pictures used mainly for random access or scene 
update. The P-pictures use forward motion predictive coding 
with reference to previously coded I- or P-pictures (anchor 
pictures), and the B-pictures use both forward and backward 
motion predictive/interpolative coding with reference to 
previously coded I- or P-picturcs. Furthermore, a group of 
picture (GOP) is formed in encoded order starting with an 
I-picture and ending with the picture before the next 
I-picture in the sequence. 

The pictures are partitioned into smaller and non- 
overlapping blocks of pixel data called Macroblocks (MBs) 
before encoding. Each MB from a P- or B-picture is sub- 
jected to a motion estimation process in which forward 
motion vectors, and backward motion vectors in the case of 
a B-picture MB, are determined using reference pictures 
from a frame buffer. With the determined motion vectors, 
motion compensation is performed where the imra-or inter- 
picture prediction mode of the MB is first determined 
according to the accuracy of the motion vectors found, 
followed by generating the necessary predicted MB. 

The predicted MB is then subjected to discrete cosine 
transform (DCT) and DCT coefficients quantization based 
on quantization matrices (QM) and quantization stcpsizc 
(QS). The quantized DCT coefficients of the MB is then 
run -length encoded with variable length codes (VLC) and 
multiplexed with additional information such as selected 
motion vectors, MB coding modes, quantization stcpsizc, 
and/or picture and sequence information, to form the output 
bilstream. 

Local decoding is performed by inverse quantizing the 
quantized DCT coefficients, followed by inverse DCT, and 
motion compensation. Local decoding is performed such 
that the reference pictures used in the motion compensation 
are identical to those to be by an external decoder. 
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The quantization stepsize (QS) used for quantizing the 
DCT coefficients of each MB has direct impact on the 
number of bits produced at the output of the VLC encoding 
process, and therefore the average output bit rate. It has also 

5 a direct impact on the encoding quality, which is output 
picture quality at the corresponding decoder. In general, 
larger QS generates lower output bit rate and lower encoding 
quality. In order to control output bit rate and picture quality 
so that the resulting bitstream can satisfy channel bandwidth 
or storage limitations as well as quality requirements, rate 
control and quantization control algorithms are used. 

Some methods for rate control and quantization control 
can be found in the abovementioned MPEG-2 TM5 (Test 
Model 5). These methods comprise generally a bit allocation 
process, a rate control process, and an adaptive quantization 

15 process. In the bit allocation process, a target number of bits 
is assigned for a new picture to be coded according to a 
number of previously determined and present parameters. 
The rate control step then calculates a reference quantization 
stepsize QS n/ for each MB based on the target bits for the 

20 picture, the number of bits already used from the target bits 
in encoding MBs from that picture, and a virtual buffer 
model as given in MPEG-2 TM5. In the adaptive quantiza- 
tion process, the calculated QS^is then scaled according to 
local activities of the MB, and an average MB activity 

25 determined from the previously coded picture. This scaling 
is done according to a level of masking effects of coding 
noise by human perception for MB with high or low 
activities within a picture. An example of an adaptive 
quantization technique is disclosed in U.S. Pat. No. 5,650, 

30 860, entitled "Adaptive Quantization". A video buffer veri- 
fier (VBV) may also be employed in such a way that 
underflow and overflow are prevented as required by the 
MPEG standard to ensure the target bit rate is maintained. 
Techniques for underflow detection and protection are also 

35 disclosed in U.S. Pat. No. 5,650,860. 

It is apparent that the fixed target bitrate in the process 
outlined above has little or no relationship to the actual or 
varying complexity of the video scenes contained in the 
input picture sequence. The target bitrate is actually defined 

40 by the communication channel bandwidth, or by the target 
storage capacity for the picture sequence, but the perceptual 
quality of the resulting pictures when decoded may vary 
from good to annoying from scene to scene according to 
scene complexity. 

45 For applications where picture sequences are compressed 
for storage and retrieval, for example DVD (Digital Video 
Disc or Digital Versatile Disc), variable bit rate (VBR) may 
be applied on individual segments of the picture sequences 
depending on its scene complexity to maximize bit rate 

50 allocation and encoded picture quality. Data bits may be 
reduced for less complex scene to save storage space and 
increase potential recording duration of the medium, or the 
resulting storage saving can be used for coding of more 
complex scenes. 

55 Similarly, VBR can also be applied to other applications 
such as a multi channel video broadcasting network. Such 
channel bandwidth may be dynamically allocated to indi- 
vidual video sequences to be multiplexed together so that 
higher percentage of the bandwidth is used adaptively by 

60 sequences with complex scenes. 

Existing VBR control algorithms such as that disclosed in 
U.S. Pat. No. 5,650360 require multiple encoding passes to 
properly distribute data bits. In the first coding pass, the bit 
utilization information is determined for each scene or each 

65 picture in the input picture sequence. This may be done by 
fixing the reference quantization stepsize and disabling the 
VBV control. 
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The determined bit utilization information is then be used 
to generate a bit budget for each scene or picture such that 
an overall target number of bits to code the sequence is fixed, 
and so that a maximum bit rate is not violated. To accom- 
plish this, the bit budget for each picture is modified so that 5 
the VBV buffer docs not underflow. In cases that initial bit 
utilization information obtained is unrealistic for generating 
the bit budget, steps from the first coding pass must be 
repeated with an adjusted reference quantization stepsize. 
The input sequence is coded in a final pass using the 10 
generated bit budget information to achieve the target bits or 
bit rate. This form of multiple-pass VBR encoder requires 
very large storage memory for storing the intermediate bit 
utilization information, and large computational capacity for 
the additional passes and the bit budget generation, is 
Furthermore, such VBR technique such as this cannot pro- 
cess the input sequence in real-time. 

SUMMARY OF THE INVENTION 

An object of the present invention is to provide a one -pass 20 
variable bit rate control technique for coding of moving 
pictures. 

In accordance with the invention, there is provided a 
method for variable bit rate control in a single pass moving 
pictures encoder, comprising: 

selecting a target picture encoding quality; 

selecting upper and lower bit rate limits; 

encoding at least one picture based on a target bit rate 
within the upper and lower bit rate limits; 

predicting a current bit rate and an encoding quality based 
on the result of the encoding step; 

comparing the encoding quality of the at least one 
encoded picture with the target picture encoding qual- 
ity; 

adjusting the target bit rate within the upper and lower bit 
rate limits according to the result of said comparison 
and the predicted current bit rate, for encoding subse- 
quent pictures; and 

repeating, for each picture in a sequence of pictures, said 
encoding, predicting, comparing and adjusting steps. 

In one form of the invention, the picture encoding quality 
is based on a mean square error. Alternatively, the picture 
encoding quality may be based on a signal-to-noisc ratio. 
Preferably the comparing step including measuring a differ- 45 
ence in picture encoding quality between corresponding 
input and locally decoded pictures. 

In a particular form of the invention, the pictures to be 
encoded are arranged in groups of pictures comprising an 
I-picture and at least one P-picture and/or B-picture, and 50 
wherein the target bit allocation is adjusted for each picture 
or plurality of pictures in each group of pictures. Preferably 
the target bit allocation is adjusted to achieve a target bit 
rale, determined on the basis of said comparison, for each 
picture or plurality of pictures in the group of pictures. 55 

The target picture encoding quality may comprise a target 
encoding quantization step-size, wherein different target 
quantization step-sizes are selected for 1-, P- and B-picturcs. 

In one form of the invention, the method may include 
measuring an average quantization step-size for at least one 60 available bits. 



25 



30 



35 



40 



tying the measured difference in picture encoding quality on 
the basis of the average picture activity. 

The present invention also provides a control apparatus 
for a single pass moving pictures encoder, comprising: 
an input for receiving a target picture encoding quality; 
an input for receiving upper and lower bit rate limits; and 
a controller for controlling the encoder so as to encode at 
least one picture according to a target bit rate within the 
upper and lower bit rate limits, predict a current bit rate 
and an encoding quality based on the result of the 
encoding, compare the encoding quality of the at least 
one encoded picture with the target picture encoding 
quality, and adjust the target bit rate according to the 
result of said comparison and the predicted current bit 
rate for encoding subsequent pictures. 
Preferably the moving picture encoder includes a fre- 
quency transform coefficient quantizer for quantization of 
the encoded picture data, and wherein the controller com- 
prises a bit rate controller coupled to control the quantization 
step size of the quantizer, a quantization step size compara- 
tor for comparing, as a measure of encoding quality, and 
actual quantization step size with a target quantization step 
size based on the target picture encoding quality, a bit 
allocation processor coupled to control the bit rate controller 
according to a number of bits remaining for encoding a 
group of pictures, and a target bit rate estimator coupled to 
receive the upper and lower bit rate limits and coupled to the 
bit allocation processor and the bit rate controller for con- 
trolling the quantization so that the required bit rate for the 
quantized picture data is within the upper and lower bit rate 
limits. 

The present invention further provides a single pass 
variable bit rate video picture encoder comprising: 
a picture input for receiving data for a plurality of moving 
pictures; 

a target quality input for receiving a target quality mea- 
sure for encoded pictures; 
an encoder output for supplying encoded picture data; 
a bit rate limit input for receiving upper and lower bit rate 

limits for the encoded picture data; 
a bit rate predictor for predicting a current bit rate; 
a frequency transform processor for frequency transform 

encoding picture data from the picture input; 
a coefficient quantize for quantizing the encoded picture 

data according to a quantization step size; 
an encoding quality estimator for measuring an encoding 

quality of quantized encoded pictures; and 
a bit rate controller for dynamically controlling the quan- 
tization step size of the coefficient quantize based on 
the predicted current bit rate and a comparison of the 
target quality and the measured quality so that the 
encoder output remains within the upper and lower bit 
rate limits. 

The video picture encoder may further include a frame bit 
counter for a number of remaining bits available for encod- 
ing a group of pictures, and a bit allocation processor for 
controlling the bit rate controller according to the remaining 



previously encoded picture, predicting a bit rate for a 
previously encoded I-, P-, and B-picture, and determining 
said target bit rate based on said predicted bit rate and a 
difference between the target encoding quantization step- 
size and the measured average quantization step-size. 

The method of the invention may include measuring an 
average picture activity for the moving pictures, and modi- 



65 



The video picture encoder may further include a quality 
comparator for comparing the target quality with a measured 
encoding quality, and wherein the bit rate controller controls 
the quantization step size based on a difference between the 
target and measured qualities. 

In one form of the encoder, the measured encoding quality 
comprises the quantization step size. 
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The video picture encoder may further include a local 
decoder for decoding the quantized encoded picture data, 
and a quality measurement processor for determining a 
quality difference between corresponding input and locally 
decoded pictures. 

In another form of the encoder, the quality measurement 
processor determines a difference in signal-to-noise ratio. 
Alternatively, the quality measurement processor may deter- 
mine a mean square error. 

It is possible in most applications that only a maximum bit 
rate and optionally a minimum bit rate are specified. Such 
applications may include a randomly accessible recording 
medium or packetized communication network with vari- 
able instantaneous bit rale but also a maximum bandwidth 
specification. In addition, such applications may also require 
that the input must be compressed in real-time, for example, 
live broadcasting or live recording cannot make use of 
multi-pass encoding. Therefore, it is also an object of the 
present invention to provide an encoder which can be 
operated in real time in a variable bit rate mode within the 
maximum and minimum bitrate boundary of the target 
application. 

While encoding an input moving picture sequence, the 
present invention continuously measures the resulting 
encoding picture quality, compares it to a defined target 
quality, and adjusts the encoding bit rale accordingly. By 
varying the target bit rate of the encoder within a defined 
maximum bit rate and a minimum bit rate according to a 
defined target encoded picture quality and the scene 
complexity, the encoder ensures consistent picture quality 
when possible and also that the maximum and minimum bit 
rate of the target application are not violated. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The invention is described in greater detail hereinafter, by 
way of example only, with reference lo the accompanying 
drawings, wherein: 

FIG. 1 is a block diagram of an MPEG compliant constant 
bit rate video encoder constructed according to the prior art; 

FIG. 2 is a block diagram of a variable bit rate moving 
pictures encoder according to one embodiment of the 
present invention; and 

FIG. 3 Is a block diagram of a moving pictures encoder 
according to an alternate embodiment of the present inven- 
tion. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

A block diagram of a generalized MPEG compliant video 
encoder 150 is illustrated in FIG. 1. A sequence of moving 
pictures (e.g. video) is input at 100 to the encoder 150 where 
it is compressed with a user defined target bitrate provided 
at 111. The target bitrate is set according to the communi- 
cation channel bandwidth in which the compressed video is 
to be transmitted, or to the storage media capacity in which 
a target duration of the compressed video is to be stored. 

Input pictures, which may be pre-processed for noise 
reduction or other purposes, are labelled as either I-pictures, 
P-pictures, or B-pictures, and stored in a frame buffer 101 
according to re-ordering and processing needs. The 
I-pictures are intra-coded pictures used mainly for random 
access or scene update. The P-pictures use forward motion 
predictive coding with reference to previously coded I- or 
P-pictures (anchor pictures), and the B-pictures use both 
forward and backward motion predictive/interpolative cod- 
ing with reference to previously coded I- or P-pictures. 
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Furthermore, a group of pictures (GOP) is formed in 
encoded order starting with an I -picture and ending with the 
picture immediately before the next I-picture in the 
sequence. In the case of an MPEG-2 encoder, the input 
5 picture sequence may be either a field or frame structure 
sequence derived for an interlaced or progressive source. 

All pictures are partitioned into smaller and non- 
overlapping blocks of pixel data called Macroblocks (MBs) 

io before actual encoding. Each MB from a P- or B-picture is 
subjected to a motion estimation process 102 in which 
forward motion vectors), ad backward motion vectors) for 
the case of a B-picture MB, are determined using reference 
picture(s) from the frame buffer 101. With the determined 

15 motion vectors), motion compensation 103 is performed 
where the intra- or inter-picture prediction mode of the MB 
is first determined according to the accuracy of the motion 
vectors) found, followed by generating the necessary pre- 
dicted MB. For the case of an I-picture MB or intra-mode 

20 MB, the predicted MB is the input MB (i.e. no prediction). 

The predicted MB is subjected to discrete cosine trans- 
form (DCT) 104 and DCT coefficient quantization 105 based 
on quantization matrices (QM) and quantization stepsize 

25 (QS). Note that for MPEG-2, field or frame based DCT can 
be selected in 104. The quantized DCT coefficients of the 
MB is then run-length encoded with variable length codes 
(VLC) at 106 and multiplexed with all necessary side 
information, for example selected motion vectors, MB cod- 

30 ing modes, quantization on stepsize, picture and sequence 
information, to form the output bitstream. The bitstrcam is 
stored in an encoder output buffer 107 for output at 108. 

Local decoding is performed by inverse quantizing the 
35 quantized DCT coefficients (109), followed by inverse DCT 
110, necessary motion compensation 103, and finally storing 
the reconstructed MB into frame buffer 101. Local decoding 
is performed such that the reference pictures used by motion 
compensation 103 is identical to those to be used by an 
40 external decoder. 

The quantization stepsize (QS) used in 118 for quantizing 
the DCT coefficients of each MB has a direct impact on the 
number of bits produced at the output of the run-length VLC 

45 encoding process 106, and therefore the average output bit 
rate. It has also a direct impact on the encoding quality, 
which is output picture quality at the corresponding decoder. 
In general, larger QS generates lower output bit rate and 
lower encoding quality. In order to control output bit rate 

50 and picture quality so that the resulting bitstream can satisfy 
channel bandwidth or storage limitations as well as quality 
requirements, rate control and quantization control algo- 
rithms are used. 

55 Previously used methods for rate control and quantization 
control can be found in the MPEG-2 TM5 (Test Model 5), 
which can be summarized in three steps. Step one is the bit 
allocation process 112 which assigns a target number of bits 
(T 7 , Tp, or T B respectively) for a new I-, P- or B-picture to 

60 be coded. The target number of bits can be computed as 
follows: 



R Bii_Ratc 
| ; NpXp | N^Xj ' 8xPicturc_Rau: 
X,K F X t K a 
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•continued 
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Tp = max 



Bit^Ralc 



Tg =max 



N B KpX B * 8 X Picture Jtate 



Bit_Rate 
NpK B X P ' 8 x Picture.Ratc 



of the picture sequences depending on its scene complexity 
to maximize bit rate allocation and encoded picture quality, 
and the bandwidth saving used for coding of more complex 
scenes, for example. However, existing VBR control algo- 
s rithms such as that disclosed in U.S. Pat. No. 5,650,860 
require multiple encoding passes to properly distribute data 
bits. Multiple-pass VBR encoder require huge storage 
memory for storing the intermediate bit utilization 
information, and large computation needs for the additional 
to passes and the bit budgets generation, and furthermore such 
techniques cannot process the input sequence in real-time. 

Before encoding an input picture sequence, an encoder 
according to an embodiment of the present invention is set 
with maximum and minimum bit rate of target application 



where 

X /= S A» Xp=SpQp, Xg-Sg.Qg, 

% S S are number of bits generated by previously ^^umlTme l^^^^^m^l^, 

encoded I-, P-, B- P icture respectively as computed by ^ a qua% fof encoded ^ ^ J larget ^ 

rate will then be set or generated. The encoder compresses 



block 114, 

Q/, Qj» Q B a re the average QS used by previously 
encoded I-, P-, B-picture respectively as computed by 
block 113, 

Bit_Rate is the Target Bitrate supplied at 111, 
Picture_Rate is number of pictures coded per second, 
K^, and K g are constants (e.g. 1.0 and 1.4 respectively), 
N P and N B are the number of P- and B-pictures remaining 
in the current GOP. 

R is the remaining number of bits assigned to the GOP, 

where R is updated as follows: 

After encoding a picture, R-R-S^^ 
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the input sequence according to this target bit rate using 
conventional bit rate control method. 

After encoding a picture or a certain number of pictures, 
the encoding quality (quality of encoded pictures) is mea- 
sured against the target. If the encoding quality is lower than 
the target quality, the target encoding bit rate will be 
increased. On the other hand, if the encoding quality is 
higher than the target quality, the larget bit rate will be 
decreased. The level of new target bit rate adjustment may 
depend on the current target bit rate and the predicted bit rate 
based on encoding history so that it is not over adjusted 



. . . . unnecessarily. The new target bit rate is also checked against 

where S. is the number of bits generated in the picture ,„ . r ,„ , ' „• „„, „■• . .„ 

. , . • , « «x 30 the denned maximum and minimum bit rate. 



just encoded (picture type is 1, P, or B) 
Before encoding the first picture in a GOP (an I-picture): 
R-G+R 

G=Bit_Rate*N/Picture_Rate 

N is the number of pictures in the GOP. 

In the second step, rate control (115) is performed. 
Basically, this step involves calculation of a reference quan- 
tization stepsize QS^for each MB based on the larget bits 
(T ; , T^ or Tfl) for the picture, the accumulated number of 



The encoder will then re-adjust its rate control system 
with the new target bit rate, and continue to encode the rest 
of the input sequence. Furthermore, the encoder can ensure 
VBV compliant with existing methods using the defined 
35 maximum bit rate and/or the target bit rate. The process of 
determining new target bit rate based on encoding quality of 
past encoded picturefs) is repeated as needed. 

In FIG. 2 a block diagram of a one-pass variable bit rate 
moving pictures encoder 250 according to one embodiment 



bits used from the target bits, and a virtual buffer model as 40 of Ihe present invention is illustrated. Those of ordinary skill 



given in MPEG-2 TM5. 

Adaptive quantisation 118 is used in the third step for 
quantization control. In this step, the calculated QS^ is 
scaled according to the local activities of the MB as gener- 
ated by an MB activity calculation process 116, and the 
average MB activity of the previously coded picture (or 
optionally current picture) as produced by a frame activity 
average process 117. This scaling is done according to a 
level of masking effects of coding noise by human percep- 
tion for MBs with high or low activities within a picture. As 
previously mentioned, an example of an adaptive quantiza- 
tion technique can be found in U.S. Pat. No. 5,650,860, 
"Adaptive Quantization". 

The frame bit-count module 114 may contain a video 
buffer verifier (VBV) which is a virtual model of an input 
buffer of an external decoder. The VBV is maintained in 
such a way that underflow and overflow are prevented as 
required by the MPEG standard to ensure target bit rate 111 
is maintained. Techniques for underflow detection and pro- 
tection may also be found in U.S. Pat. No. 5,650,860. 

As mentioned, this form of encoding does not take 
account of the actual or varying complexity of the video 
scenes contained in the input picture sequence 100. 
Consequently, the perceptual quality of the resulting pictures 
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in the art will recognise that a number of the functional 
blocks illustrated are common with the conventional MPEG 
encoder described above. The input moving picture 
sequence 200 is processed picture by picture according to its 
possible assigned picture coding type (I-, P-, of B-picturc). 
A frame buffer 201 is used to temporarily store the input 
pictures according to re-ordering and processing needs. 
Each input picture is divided into Macroblocks (MB) before 
encoding. Temporal redundancy of each MB may be 
removed by motion estimation 202 and motion compensa- 
tion 203. 

After necessary motion compensation, a MB is subjected 
to discrete cosine transform (DCT) 204 and DCT coefficient 
quantization 205 based on quantization matrices and quan- 
tization stepsize. The quantized MB of I- or P-picture is 
inverse quantized 209, inverse discrete cosine transformed 
210, and subjected to corresponding motion compensation 
203 so that the quantized MB is reconstructed and stored 
locally in frame buffer 201 for future motion estimation and 
compensation needs. The quantized MB is run-length 
encoded with VLC (variable length coding) at 206, together 
with all necessary side information to form the encoded 
bitstream of the input sequence. The encoded bitstream is 
stored in an output buffer 207 of the encoder for output at 



may vary from good to annoying from scene to scene 65 208 at desired data rates, 
according to scene complexity. Variable bit rate (VBR) Methods of motion estimation 202, motion compensation 
coding may be therefore be applied on individual segments 203, DCT 204, DCT coefficient quantization 205, run-length 
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VLC encoding 206, inverse quantization 209, and inverse 

DCT 210 may be those defined and/or allowed in the where K P and K fl are constants which can be experimen- 

1SO/IEC MPEG-1 or MPEG-2 standards. tally determined (example K P -1.0 and K B -\A). 

Before encoding, a target cuality of encoding (219) and a further condition may be applied to the final QS^^ for 

maximum/minimum bit rate (211) arc set. The maximum 5 B-picrures such that it is not lower than the QS ov>eragt . of the 

and minimum bit rates (BR^ and BR,^, given as bits per last coded 1- or P-picture so that bits can be efficiently used 

second, define the boundary bit rates which the encoder shall to improve the quality of anchor I- or P-pictures first before 

operate at, and the target quality defines the encoded picture improving the quality of the B-picrures. 

quality thai the encoder shall target by constantly adjusting i n the process of Frame Quality Measure 220 process, the 

a target bit rate within the given bit rate boundaries. The 10 target quality QS , is compared to the picture quality 

minimum bit rate may be set to zero. QS aw ^ e . Since a higher value of QS av€mge implies higher 

Initially, target bit rate estimation 223 may be performed quantization noise and therefore lower encoded picture 

based the set target quality and the max/min bit rates to quality, when the value of QS^,.^ is found to be higher 

generate an initial target bit rate (BR^J. Such initial than QS ta ^ en the difference of the two values will be used 

estimate may be based on experimentally determined fixed 15 t 0 increase the target bit rate (BR WJS<rt ) at target bitrate 

values which may not directly relate to the picturefs) to be estimator 222. On the other hand, when QS^^ is lower 

coded. With the target bit rate, the bit allocation 212 is than QS to ^ rt , BR WnfrJ is decreased at target bitrate estimator 

performed to determine a target number of bits for the 222. Optionally, the value of QS„ , may be used at the rate 

picture to be coded (T /fP>b ). A rate controller 215 determines controller 215 as a lower limit for the final output QS,,,, 

a reference quantization step size (QS^) for each MB based 20 value so that when the target is reached (target quality 

on the estimated number of bits for the picture (T /Ai> ) and reached), bits are saved for future encoding immediately, 

the bit utilization by the run-length VLC encoder 206. The Hence, the output reference quantization stepsize at rate 

QS re/ of each MB in the picture to be coded can be controller 215 may be set according to: 
computed, for example, as follows: 

25 

Dw xKt if (QS,^QS to ,^), then QS^S^. 

(2xBfl w/ rt/picuirc_raic) For target bit rale estimation, a rate-quantization model 

for example one developed by Wei Ding and Bede Liu, 

w jj ere "Rate Control of MPEG Video Coding and Recording by 

. . , „ , 30 Rate-Quantization Modelling", IEEE Trans, on Circuit and 

D w is virtual buffer fullness of corresponding I- P-, or Systems for Vld(jQ Tecrm ology, Vol. 6, No. 1, February 1996, 

B-picture, updated (after coding each MB) by the be ad d To avoid , error or lex 

difference between the bits used by the MB and the bits local fitUn of the ^-quantization model, an alternative 

allocated to the MB based on the corresponding T^j* embodiment of the present invention may consist of a 

a set of initial values for D B may be assumed at 35 method of target bitrate estimation comprising the steps of 

sequence start, and predicting a bitrate (BR prwUa ^ at a Bitrate Predictor 221 

K 7 is a constant (eg. 31). based on bits used for encoding the last I-, P-, and B-picture, 

Adaptive quantization 218 may be applied in which the and estimating a new target bitrate (BR Wf?rt ) at the target 

determined QS^is scaled according to the local activities of bitrate estimator 222 based on the said predicted bit rate and 

the MB as generated by MB activity calculation process 216 40 the difference between QS average and QS^^,,. The predicted 

and the average MB activity of the previously coded (or bitrate (BR preMel J^ and the new target bitrate (BR M ^ ( ) may 

optionally current picture) as produced by the frame activity be computed as follows: 
average process 217. Example implementations of the MB 

activity calculation 216, frame activity average 217, and _ {S, +nf.xS P +n B xS B ) 

adaptive quantization process 218 arc found in MPEG-2 45 (i + 

TM5. The output quantization stepsize (QS) is used to ^q S _q S j 

quantized DCT coefficients of the MB. BR nri „ = br^* + x — =^ 

After encoding a picture, the number of bits (S,^, " mtt 
corresponding value for I-, P-, or B-picture) generated by 

encoding the picture is accumulated by Frame Bit-Count 50 

214, and the quantization stepsize (QS) is averaged by (BR^BR^J, then br^bfu, 

Frame Q Step Average 213 process. The encoded picture if (BR.^^br.j, then br^^-br^^, 

quality is also determined by Frame Quality Measure 220 w here 

process. One method of determining the encoded picture ooo • . 

quality is by the average value of the reference quantization 55 S„ S fl are number of bits generated by previously 

stepsize (QS^,) used for coding the picture since it enc ° dcd l ~> P "' B /P Ict " rc ' . . u 

indicates roughly'the amount of quantization noise in the °p 15 lhe lolal Qumber of Pictures in the current GOP, 

encoded picture. In this method, the target quality set at 219 n a k lhe 10131 number of B-pictures in the current GOP, 

is defined as the target reference quantization stepsize an d 

(0S Mniei ). Three different QS M ^, values may be set at 219 <® 1^ may be a constant, or a factor of BR^^ BR^ or 
for corresponding I-, P-, and B-pictures; alternatively, the 

values can be determined by: The new target bit rate (BR tonJ „) can be computed after 

encoding each picture or after encoding a certain number of 

OS Mvt for i-pictures-os^,^, pictures. The bit allocation process 212 and the rate con- 

os for P ictur«-K *os 65 tr0 ^ er are u P^ atwl wth the new BR te _., value once it 

-P 1 w- f •">« ^ determined. An embodiment of the bit allocation process 

Uiv,^ for B-picturcs-Kfl'QS,.,,,, 212 comprises the steps of: 
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a) Before encoding the first picture in a group of pictures according to the present invention. In this embodiment, the 
(GOP), determining R which is the remaining number of encoding quality 324 is determined by comparing the origi- 
bits assigned to the GOP: oal input picture taken from input 300 and the locally 

decoded picture 325. Conventional methods such as the 
r +- N xbr then set BR*s -BR 5 mean SRU&K error (MSE) or the signal-to-noise ratio (SNR) 

~ picture_nue J " wt ~ measures can be used as comparison. This measure is further 

compared at Frame Quality Measure 320 with the definable 
wnere . Target Quality 319. When the encoding quality is lower than 

VI . ' , - . . ___ the target quality, the target bitrate estimator 322 will set the 

N is the number of pictures in the GOP „ , . u-. . u-u .u .u u j -c .u j- 

r 10 target bitrate higher, on the other hand, if the encoding 

R-0 before start of sequence, and quality b man ^ targe , ^ larget bilfate ^ 

+-is the accumulate function ^ lower. 

b) Otherwise, if the picture to be coded is not the first picture ^ j nput pj clure or a g j ve n number of input pictures are 
of a GOP and a new target bit rate (BR,^,) is first encoded with an initial set of encoding parameters. Each 
determined, then updating R as given by: 15 0 f ^sc pictures may be given a picture coding type of either 

I-, P-, or B-picture. The encoding technique such as those 

iNp+N B ) }i defined by the MPEG-1 or MPEG-2 standards may be 

pjeturcjate utilized. Such techniques include motion estimation 302, 

then $« br^ ^ = BR unf , motion compensation 303, discrete cosine transform 304, 

20 DCT coefficient quantization 305, and run-length VLC 
encoding 306 for temporal, spatial, and statistical redun- 

WDere: dancy reduction. A frame buffer 301 is used for storing the 

N,, and N fl are the number of P- and B-pictures remaining input pictures for re-ordering and processing needs, 

in the current GOP Before each picture is coded, its target amount of bits is 

c) Obtaining the average quantization stepsize (Q„ Q^, or 2 s set by bit allocation process 312 based on the initial set of 
Q B ) from previously encoded pictures (I-, P-, or B-picture coding parameters as well as any updated information from 
respectively) as determined by Frame Q Step average Frame Q Step averaging process 313 and Frame bit-count 
process 213, 314. Rate Controller 315 determines reference quantization 

d) Obtaining from frame bit-count 214 the value of the stepsize for each MB while coding the picture. Conventional 
number of bits (S /5 S^ or S^) used to encode the past I-, 30 adaptive quantization 318 which depends on MB activity 
P-, or B-pictures, and removing number of bits used by calculation 316 and frame activity average 317 can be 
the last coded picture (S,^) from R, hence: applied to each reference quantization stepsize to generate 

final quantization stepsize for the DCT Coeff quantizer 305. 
R -=s t if last coded picture is an /-picture The coded pictures are locally decoded by inverse quan- 

35 tization 309, inverse discrete cosine transform 310, inverse 
R -= s P if last coded picture is an P-picture of me molion compensation 303 to form the locally decoded 

R -= s B if last coded picture is an B- picture pictures 325. It is stored in the frame buffer 301 if necessary 

for future prediction use (ie. anchor pictures). The locally 
decoded pictures 325 are used with reference to the original 

e) Finally, determining the target number of bits (T„ Tp, or 40 input pictures 300 for encoding quality determination 324. 
T g ) to be used for coding the current picture. T /t T,,, or T fl Th e determined encoding quality is compared to the target 
for respective I-, P-, or B-picture are given, for example, quality given by 319 at frame quality measure process 320. 
by: It is also possible to offset the determined encoding quality 

by the picture activity (ACT av€rage ) before the comparison 

T) * 45 so as to take into account the masking effects of the human 

j + NpXp + N * x * visual system to the average level of activity within the 

x,k p x,k b picture. For example if the encoding quality is determined 

# by the peak signal-to-noise ratio, then the resulting encoding 

Tp = NbKrXb quality difference (SNR rf ^) at the output of frame quality 

Np + K B Xp 50 measure 320 can be given by: 



° K ' X * 

where 

55 

where 

XrSjQ,, X F SrQp, X B -S fl ,Q a , ^ isTconstant (eg. o"oi8),'and 

K„ and K fl are constants. A/ -_, ... . , ... 

* . . B . _ . . ... . ACT is the average MB luminance block variance. 

A minimum value (T^,) may be set as the lower bound- Qne meli f od of adjustin BR bascd 0D SNR^at the 

ary for T„ T„ T,. For example: largc , bitrate estimator 322 ^ by . ' 

j. = BRt ^" BR^^BR^^K^BR^^SNR,^ 

K } xPicture_Raie" 

FIG. 3 is a block diagram illustrating another embodiment & * m »'*" <BR -* ) - BR -^- BR «-*. 

of a one-pass variable bit rate moving pictures encoder 350 where 
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^predicted is 'be result of bilrate predictor 321, 
W&currrnt »s the previous value of BR, fl/xr , stored at current 

bitrate 323 memory, 
K« is a suitable constant, 

BR m/u . and BR^ arc the maximum and minimum bitrate 
set at 311. 

The bit allocation module 312 and the rate controller 315 
take the updated BR M/XW from the target bitrate estimator 
322 and perform necessary bit allocation and bit rale control 
such that the resulting compressed moving picture bit- 
streams will have encoded bit rate close to BR M ^ <r Existing 
techniques of VBV (video buffer verifier) underflow detec- 
tion and protection may be applied using BR^ as the 
reference bit rate to ensure the maximum bitrate of the 
output bitstream is not violated. 

Because the target bit rate BR m#sw for encoding is adapted 
based on feedback on encoding quality, the encoding quality 
will always converge to a defined target quality. In such a 
way the encoding accommodates variable bit rates in real- 2Q m „ t£) . 
time within single computational pass, and the encoding 
process can be better tuned to changes in moving pictures 
scene content complexity. The overall perceptual quality of 
the encoded moving pictures sequence will be relatively 
consistent with reference to the defined target. 

Embodiments according to the present invention can 
accommodate simplified models for estimation of target bit 
rate based on feedback and prediction. Overall impact to 
implementation and computation complexity as well as 
memory storage requirements of the encoder is minimized. 3Q 

The foregoing detailed description of embodiments of the where: 
invention has been presented by way of example only, and 
is not intended to be considered limiting to the invention as 
defined in the claims appended hereto. 

What is claimed is: 

1. A method for variable bit rate control in a single pass 
moving pictures encoder, comprising: 

selecting a target picture encoding quality; 

selecting upper and lower bit rate limits; 

encoding at least one picture based on a target bit rate 40 
within the upper and lower bit rate limits; 

predicting a current bit rate and an encoding quality based 
on the result of the encoding step; 

comparing the encoding quality of the at least one 
encoded picture with the target picture encoding qual- 
ity; 

adjusting the target bit rate within the upper and lower bit 
rate limits according to the result of said comparison 
and the predicted current bit rate, for encoding subse- 
quent pictures; and 

repeating, for each picture in a sequence of pictures, said 
encoding, predicting, comparing and adjusting steps. 

2. A method as claimed in claim I, wherein the step of 
encoding at least one picture based on the target bit rate 
includes determining a target bit allocation for the at least 
one picture using the target bit rate. 

3. A method as claimed in claim 2, wherein the pictures 
to be encoded are arranged in groups of pictures comprising 
an 1-picture and at least one P-picture and/or B-picture, and 
wherein the target bit allocation is adjusted for each picture 
or plurality of pictures in each group of pictures. 

4. A method as claimed in claim 1, 2 or 3, wherein the 
picture encoding quality is based on encoding quantization 
step-size. 

5. A method as claimed in claim 1, 2 or 3, wherein the 
picture encoding quality is based on a mean square error. 



6. A method as claimed in claim 1, 2 or 3, wherein the 
picture encoding quality is based on a signal-to-noise ratio. 

7. A method as claimed in claim 5 or 6, wherein the 
comparing step includes measuring a difference in picture 
encoding quality between corresponding input and locally 
decoded pictures. 

8. A method as claimed in claim 7, including measuring 
an average picture activity for the moving pictures, and 
modifying the measured difference in picture encoding 
quality on the basis of the average picture activity. 

9. A method as claimed in claim 8, including measuring 
a signal-to-noise ratio for at least one previously encoded 
picture, and determining said target bit rate based on said 
predicted current bit rate and on a difference between the 
measured signal-to-noise ratio and a target signal-to-noise 
ratio as said target picture encoding quality. 

10. A method as claimed in claim 9, wherein the predicted 
current bit rate and the target bit rate are determined accord- 



(S, +n P xS P +n B xS B ) 
(1 + n? + n B ) 
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SNRp/ctBr* is the measured signal-to-noise ratio, 
SNR MFg<f is the target signal-to-noise ratio, 
ACT av*«>s« ^ the measured average pixel activity, 
Kj and are constants, 

SNR^y is the measured difference in picture encoding 
quality, 

^predicted is the predicted current bit rate, 
S p Sp, and S g are number of bits generated by said 

previously encoded I-, P-, B-pictures, respectively 
n P is a total number of P-pictures in the group of pictures, 
n a is a total number of B-pictures in the group of pictures, 
BR CM ^ W is a previously determined BR te ^, n and 
^target 15 target bit rate. 

11. A method as claimed in claim 4, wherein said target 
picture encoding quality is based on a target encoding 
quantization step-size. 

12. A method as claimed in claim 11, wherein different 
target quantization step-sizes are selected for I-, P-, and 
B-pictures. 

13. A method as claimed in claim 11 or 12, including 
measuring an average quantization step-size for at least one 
previously encoded picture, and determining said target bit 
rate based on said predicted current bit rate and a difference 
between the target encoding quantization step-size and the 
measured average quantization step-size. 

14. A method as claimed in claim 13, wherein maximum 
and minimum bit rates are selected corresponding to the 
upper and lower bit rate limits, and wherein the predicted 
current bit rate and the target bit rate are determined accord- 
ing to: 



{Si+n F xS P +n B x.S B ) 
(1 +n P + n B ) 
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-continued 
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-continued 



BR tuxet =BR prtdiaili +K 2 x 



if (BR^^BR^X *e» BR^-BR,.., 
if (BR^^BR^), then BR^ /ptt -BR^ 



_R 

K B X P 



T B = . 



K,X B 



where 



^pmdicud ^ s *id predicted current bit rate, 
BR target is said target bit rate, 



10 where: 

X r SjQ n Xf.S P Qr, X B S B ,Q B , 
K p and K g are constants. 

17. A method as claimed in claim 16, wherein the target 



15 
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BR^^ is said maximum bit rate, 
BR™* is said minimum bit rate, 

S /( Sp, S B are number of bits generated by said previously 

encoded I-, P-, B-pictures, respectively, 
n^, is a total number of P-pictures in the group of pictures, 
n B is a total number of B-pictures in the group of pictures, 

and 

K 2 is a constant or a factor of BR^^^, BR^^ or 

previous BR Wf . 
15. A method as claimed in claim 10 or 14, including 25 
determining a number of bits R remaining for encoding the 
group of pictures comprising the steps of: 

before encoding a first picture in a said group of pictures, 
determining R according to 



picture_rate 



xBR,^, then set «Uu«*» = BR ^,r, 



where: 

N is the number of pictures in the group of pictures, 
picture_rate is the number of pictures encoded per unit 

time, and 
BR larget is said target bitrate, 

and before encoding each subsequent picture in the group of 

pictures, determining R according to 



30 



35 



40 



R += — — X (BR a „„ — BR git tMm)' 



45 



-5. then set BR M ufm = BR^,^ 



where: 

Nj> and N fl are the number of P- and B-pictures remaining 

in the current group of pictures, 
BR f4If¥W is a subsequently determined target bit rate, and 
S is one of S,, S P or S B according to the picture type of 

the previously encoded picture. 
16. A method as claimed in claim 15, further comprising 
the steps of: 

obtaining average quantization step sizes Q It Q^, Q B from 
previously encoded I-, P-, B-pictures, respectively; and 

determining the target bit allocation T 7 , Tp, T B for a 
subsequent respective I-, P-, or B-picture according to: 



50 



55 



60 



according to: 



Kj xpicture_ratc 



l | N P Xr t N B X B 
X/Kp X t K B 



where K 3 is a constant. 

18. A method as claimed in any one of claims 1 to 17, 
wherein the target bit allocation is used to encode an input 
picture, comprising the steps of: 

deriving macroblocks from the input picture; 
calculating a quantization step-size for each said macrob- 
lock according to the target bit allocation and its 
utilization; 

performing motion compensation if necessary to each 
macroblock; 

frequency transforming each said macroblock or motion 

compensated macroblock; 
quantizing each transformed macroblock using the calcu- 
lated quantization step-size; and 
coding the quantized macroblock. 

19. A method as claimed in claim 18, wherein the target 
quantization step-size is used as a lower limit for the 
determined quantization step-size. 

20. A method as claimed in claim 18 or 19, wherein the 
determined quantization step -size for each macroblock is 
modulated by a corresponding macroblock activity masking 
factor, and the quantization is performed using the modu- 
lated quantization step-size. 

21. A control apparatus for a single pass moving pictures 
encoder, comprising: 

an input for receiving a target picture encoding quality; 
an input for receiving upper and lower bit rate limits; and 
a controller for controlling the encoder so as to encode at 
least one picture according to a target bit rate within the 
upper and lower bit rate limits, predict a current bit rate 
and an encoding quality based on the result of the 
encoding, compare the encoding quality of the at least 
one encoded picture with the target picture encoding 
quality, and adjust the target bit rate according to the 
result of said comparison and the predicted current bit 
rate for encoding subsequent pictures. 

22. A control apparatus as claimed in claim 21, wherein 
encoding the at least one picture based on the target bit rate 
includes determining a target bit allocation for the at least 
one picture using the target bit rate. 

23. A control apparatus as claimed in claim 22, wherein 
the pictures to be encoded are arranged in groups of pictures 
comprising an I-picture and at least one P-picture and/or 

65 B-picture, and wherein the controller adjusts the target bit 
allocation for each picture or plurality of pictures in each 
group of pictures. 
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24. A control apparatus as claimed in claim 21, 22 or 23, 
wherein the picture encoding quality is based on encoding 
quantization step-size. 

25. A control apparatus as claimed in claim 21, 22 or 23, 
wherein the picture encoding quality is based on a mean 
square error. 

26. A control apparatus as claimed in claim 21, 22 or 23, 
wherein the picture encoding quality is based on a signal- 
to-noise ratio. 



target quality and the measured quality so that the 
encoder output remains within the upper and lower bit 
rate limits. 

32. A video picture encoder as claimed in claim 31, 
wherein the pictures to be encoded are arranged in groups of 
pictures comprising an I-picture and at least one P-picture 
and/or B-picture, the bit rate controller including a frame bit 
counter for a number of remaining bits available for encod- 
ing a group of pictures, and a bit allocation processor for 



27. A control apparatus as claimed in claim 25 or 26, 10 controlling the bit rate controller according to the remaining 

available bits so as to adjust a target bit allocation for each 
picture or plurality of pictures in each group of pictures. 

33. A video picture encoder as claimed in claim 31 or 32, 
including a quality comparator for comparing the target 

15 quality with the measured encoding quality, and wherein the 
bit rate controller controls the quantization step size based 
on a difference between the target and measured qualities. 

34. A video picture encoder as claimed in claim 32, 
including a quality comparator for comparing the target 

20 quality with the measured encoding quality wherein the bit 
rate controller adjusts the target bit allocation to achieve a 
target bit rate for the picture or plurality of pictures in each 
group of pictures which is determined on the basis of an 
output from said quality comparator. 
25 35. A video picture encoder as claimed in claim 31, 32, 33 
or 34, wherein the measured encoding quality is based on the 
quantization step size. 

36. A video picture encoder as claimed in claim 33, 
including a local decoder for decoding the quantized 



wherein the controller includes a local picture decoder and 
a comparator for measuring a difference in picture encoding 
quality between corresponding input and locally decoded 
pictures. 

28. A control apparatus as claimed in claim 27, wherein 
the controller includes a picture activity processor for mea- 
suring an average picture activity for the moving pictures, 
and wherein the controller modifies the measured difference 
in picture encoding quality on the basis of the average 
picture activity. 

29. A control apparatus as claimed in claim 28, wherein 
the controller includes a signal-to-noise processor for mea- 
suring a signal-to-noise ratio for at least one previously 
encoded picture, and a bit rate predictor for predicting the 
current bit rate, and wherein the controller adjusts said target 
bit rate based on the predicted current bit rate and on a 
difference between the measured signal-to-noise ratio and a 
target signal-to-noise ratio as said target picture encoding 
quality. 



30. A control apparatus as claimed in claim 21, wherein 30 encoded picture data, and a quality measurement processor 

the moving picture encoder includes a frequency transform ror determining a quality difference between corresponding 

coefficient quantizer for quantization of the encoded picture m P ul and locally decoded pictures, 

data, and wherein the controller comprises a bit rate con- 37. A video picture encoder as claimed in claim 36, 

troller coupled to control the quantization step size of the wherein the quality measurement processor determines a 

quantizer, a quantization step size comparator for 35 difference in signal-to- noise ratio, 

comparing, as a measure of encoding quality, an actual 38 - A video picture encoder as claimed in claim 36, 

quantization step size with a target quantization step size wherein the quality measurement processor determines a 

based on the target picture encoding quality, a bit allocation mean square error. 

processor coupled to control the bit rale controller according 39. A video picture encoder as claimed in claim 36, 37 or 

to a number of bits remaining for encoding a group of 40 38, including a picture activity processor for measuring an 



pictures, and a target bit rate estimator coupled (o receive the 
upper and lower bit rate limits and coupled to the bit 
allocation processor and the bit rate controller for control- 
ling the quantization so that the required bit rate for the 
quantized picture data is within the upper and lower bit rate 45 
limits. 

31. A single pass variable bit rate video picture encoder 
comprising: 

a picture input for receiving data for a plurality of moving 
pictures; 

a target quality input for receiving a target quality mea- 
sure for encoded pictures; 
an encoder output for supplying encoded picture data; 



50 



average picture activity for the moving pictures, and wherein 
the quality measurement processor modifies the measured 
difference in picture encoding quality on the basis of the 
average picture activity. 

40. A video picture encoder as claimed in claim 37, 
including a signal-to-noise processor for measuring a signal- 
to-noise ratio for at least one previously encoded picture, 
and a bit rate predictor for estimating a current bit rate, and 
wherein the bit rate controller adjusts said target bit alloca- 
tion based on the estimated current bit rate and on a 
difference between the measured signal-to-noise ratio and a 
target signal-lo-noise ratio as said target picture encoding 
quality. 

41. A video picture encoder as claimed in claim 31, 



a bit rate limit input for receiving upper and lower bit rate 55 including a quantization step size comparator for comparing, 



limits for the encoded picture data; 
a bit rate predictor for predicting a current bit rate; 
a frequency transform processor for frequency transform 

encoding picture data from the picture input; 
a coefficient quantizer for quantizing the encoded picture 

data according to a quantization step size; 
an encoding quality estimator for measuring an encoding 



as a measure of encoding quality, an actual quantization step 
size with a target quantization step size based on the target 
picture encoding quality, a bit allocation processor coupled 
to control the bit rate controller according to a number of bits 
50 remaining for encoding a group of pictures, and a target bit 
rate estimator coupled to receive the upper and lower bit rate 
limits and coupled to the bit allocation processor and the bit 
rate controller for controlling the quantization so that the 
required bit rate for the quantized picture data is within the 



quality of quantized encoded pictures; and 
a bit rate controller for dynamically controlling the quan- 65 upper and lower bit rate limits, 
tizalion step size of the coefficient quantizer based on 
the predicted current bit rate and a comparison of the 
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